A global goodness-of-fit test for receiver operating characteristic curve analysis via the bootstrap method

نویسندگان

  • Kelly H. Zou
  • Frederic S. Resnic
  • Ion-Florin Talos
  • Daniel Goldberg-Zimring
  • Jui G. Bhagwat
  • Steven Haker
  • Ron Kikinis
  • Ferenc A. Jolesz
  • Lucila Ohno-Machado
چکیده

OBJECTIVE Medical classification accuracy studies often yield continuous data based on predictive models for treatment outcomes. A popular method for evaluating the performance of diagnostic tests is the receiver operating characteristic (ROC) curve analysis. The main objective was to develop a global statistical hypothesis test for assessing the goodness-of-fit (GOF) for parametric ROC curves via the bootstrap. DESIGN A simple log (or logit) and a more flexible Box-Cox normality transformations were applied to untransformed or transformed data from two clinical studies to predict complications following percutaneous coronary interventions (PCIs) and for image-guided neurosurgical resection results predicted by tumor volume, respectively. We compared a non-parametric with a parametric binormal estimate of the underlying ROC curve. To construct such a GOF test, we used the non-parametric and parametric areas under the curve (AUCs) as the metrics, with a resulting p value reported. RESULTS In the interventional cardiology example, logit and Box-Cox transformations of the predictive probabilities led to satisfactory AUCs (AUC=0.888; p=0.78, and AUC=0.888; p=0.73, respectively), while in the brain tumor resection example, log and Box-Cox transformations of the tumor size also led to satisfactory AUCs (AUC=0.898; p=0.61, and AUC=0.899; p=0.42, respectively). In contrast, significant departures from GOF were observed without applying any transformation prior to assuming a binormal model (AUC=0.766; p=0.004, and AUC=0.831; p=0.03), respectively. CONCLUSIONS In both studies the p values suggested that transformations were important to consider before applying any binormal model to estimate the AUC. Our analyses also demonstrated and confirmed the predictive values of different classifiers for determining the interventional complications following PCIs and resection outcomes in image-guided neurosurgery.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation

This review provides the basic principle and rational for ROC analysis of rating and continuous diagnostic test results versus a gold standard. Derived indexes of accuracy, in particular area under the curve (AUC) has a meaningful interpretation for disease classification from healthy subjects. The methods of estimate of AUC and its testing in single diagnostic test and also comparative studies...

متن کامل

Diagnostic evaluation of milk lactate dehydrogenase and alkaline phosphatase activities by receiver operating characteristic analysis curve in early lactation of ewes with subclinical mastitis

The aim of the present study was to investigate the diagnostic evaluation of milk lactate dehydrogenase (mLDH) and alkaline phosphatase (mALP) activities by receiver operating characteristic (ROC) analysis curve in early lactation of ewes with subclinical mastitis (SCM) and determine the correlation between number of somatic cell count (SCC) and mLDH and mALP activities. A total of 196 udder ha...

متن کامل

Development and Validation of Web-Based Nomograms to Predict Postoperative Invasive Component in Ductal Carcinoma in Situ at Needle Breast Biopsy

OBJECTIVES Although sonography-guided core needle biopsy is a highly targeted method, there is a possibility of an invasive component after surgical excision of ductal carcinoma in situ (DCIS) of the breast. This study was performed to develop and validate nomograms to predict the postoperative invasive component in DCIS at core needle biopsy. METHODS Two nomograms were developed using the da...

متن کامل

Application of adjusted-receiver operating characteristic curve analysis in combination of biomarkers for early detection of gestational diabetes mellitus

Introduction: In medical diagnostic field, evaluation of diagnostic accuracy of biomarkers or tests has always been a matter of concern. In some situations, one biomarker alone may not be sufficiently sensitive and specific for prediction of a disease. However, combining multiple biomarkers may lead to better diagnostic.  The aim of this study was to assess the performance of combination of bio...

متن کامل

Regression Analysis for the Partial Area under the Roc Curve

Performance evaluation of any classification method is fundamental to its acceptance in practice. Evaluation should consider the dependence of a classifier’s accuracy on relevant covariates in addition to its overall accuracy. When developing a classifier with a continuous output that allocates units into one of two groups, receiver operating characteristic (ROC) curve analysis is appropriate. ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • Journal of biomedical informatics

دوره 38 5  شماره 

صفحات  -

تاریخ انتشار 2005